Skip to content

fix: handle DECIMAL(38,9) type conversion in BigQuery processor#582

Closed
devin-ai-integration[bot] wants to merge 7 commits intomainfrom
devin/1737758508-fix-bigquery-decimal-type-handling
Closed

fix: handle DECIMAL(38,9) type conversion in BigQuery processor#582
devin-ai-integration[bot] wants to merge 7 commits intomainfrom
devin/1737758508-fix-bigquery-decimal-type-handling

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration bot commented Jan 24, 2025

Fix BigQuery DECIMAL(38,9) type conversion

This PR fixes the issue where BigQuery cache operations fail with "Invalid value for type: DECIMAL(38, 9)" error by properly mapping SQLAlchemy DECIMAL types to BigQuery's NUMERIC type with appropriate precision and scale parameters.

Changes:

  • Added explicit handling of DECIMAL types in BigQueryTypeConverter
  • Maps SQLAlchemy DECIMAL to BigQuery NUMERIC(38,9)

Testing:

  • Manual testing needed to verify the fix works with actual BigQuery cache operations

Fixes #573

Link to Devin run: https://app.devin.ai/sessions/a5b1089b2e6749039d54de93a56e7c39

@devin-ai-integration @aaronsteers
fix: handle DECIMAL(38,9) type conversion in BigQuery processor
0b1c2b1 
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

devin-ai-integration bot commented Jan 24, 2025

Original prompt:

@Devin - Can you investigate this one? Per the reporters most recent comment, we should be able to use the pre- and post-manifesting version diff for clues. <https://github.com/airbytehq/pyairbyte/issues/573>

@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

devin-ai-integration bot commented Jan 24, 2025

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add "(aside)" to your comment to have me ignore it.
  • Look at CI failures and help fix them

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@devin-ai-integration devin-ai-integration bot mentioned this pull request Jan 24, 2025
Open
@devin-ai-integration @aaronsteers
fix: handle DECIMAL(38,9) type conversion in BigQuery processor
9caefb4 
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
@aaronsteers Aaron ("AJ") Steers (aaronsteers) marked this pull request as ready for review January 24, 2025 22:58
devin-ai-integration bot and others added 5 commits January 24, 2025 23:12
@devin-ai-integration @aaronsteers
test: improve BigQuery decimal type test cleanup
ca32627 
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
@devin-ai-integration @aaronsteers
test: improve BigQuery decimal type test
b13538c 
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
@devin-ai-integration @aaronsteers
style: format test_bigquery_cache.py with ruff
05d5e53 
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
@devin-ai-integration @aaronsteers
fix: use execute_sql instead of _execute_sql in BigQuery test
3042f56 
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
@devin-ai-integration @aaronsteers
test: ensure schema exists before creating BigQuery test table
f35332e 
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

devin-ai-integration bot commented Feb 2, 2025

Closing due to inactivity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aaronsteers Aaron ("AJ") Steers (aaronsteers) Awaiting requested review from aaronsteers

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Cache BigQuery: Invalid value for type: DECIMAL(38, 9) is not a valid value

0 participants